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(54) Image data compression method and system 



(57) A method of image compression includes digi- 
tizing an image and segmenting the image in a plurality 
of different manners to generate a plurality of segment- 
ed images. Each of the segmented images is com- 
pressed. The method further includes determining a bit 



rate for each of the compressed images, and determin- 
ing how much image distortion results from each com- 
pression, Finally, the manner of segmentation which re- 
sults in an optimal compromise between the rate and 
distortion is selected. 
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Description 

[0001] The present invention relates to the art of im- 
age processing. It finds particular application in conjunc- 
tion with digital image processing for file compression, 5 
and will be described with particular reference thereto. 
However, it is to be appreciated that the present inven- 
tion is also amenable to other like applications. 
[0002] In a modern office environment, it is common 
to have many documents digitally scanned, electroni- 
cally created, stored, transmitted, printed and/or dis- 
played. Typically, it is preferred that these operations be 
performed rapidly. Nevertheless, user expectations of 
quality are still often high. Digital implementation of a 
rapid high quality image path can be particularly formi- 
dable considering that a single page of a color document 
scanned at 600 spots per inch (spi) may be approxi- 
mately 1 00 Megabytes in size. Consequently, practical 
systems for processing color or other sizable docu- 
ments demand document compression methods that 
achieve high compression ratios with low distortion. 
[0003] "Document" images generally differ from "nat- 
ural" images because they tend to contain well defined 
regions with distinct characteristics, such as text, graph- 
ics, continuous-tone pictures, halftone pictures and 
background. For example, typically, it is desired that text 
have a high spatial resolution for legibility, while high 
color resolution is often not required. Alternatively, con- 
tinuous-tone pictures benefit from high color resolution, 
but can tolerate relatively lower spatial resolution. 
Therefore, It is desirable that a document compression 
algorithm be adaptive in order to meet different goals 
and exploit different types of redundancy among differ- 
ent image classes or types. Nevertheless, traditional 
compression algorithms, such as JPEG, are based on 
the assumption that an input image is spatially homoge- 
neous, so they tend to perform poorly on document im- 
ages. 

[0004] A commonly used format in which images are 
represented for document or image compression is the 
known 3-layer model based foreground/mask/back- 
ground representation for mixed raster content (MRC). 
Generally, the foreground layer contains the text and 
line graphics, and the background layer contains pic- 
tures and background. The mask is a binary image 
which determines, for each pixel in the digitized image, 
if the foreground pixel information or the background 
pixel information should be used. To apply the 3-layer 
MRC model, a document image is first segmented into 
foreground and background layers, and an appropriate 
mask is generated. 

[0005] The subsequent performance of a document 
or image compression system or algorithm is directly re- 
lated to the segmentation. With respect to document or 
image compression, an advantageous segmentation 
not only lowers the bit rate of the compressed image (i. 
e.,the number of bits used to represent the compressed 
image per pixel in the uncompressed image), but also 



lowers the distortion in the reconstructed image. On the 
other hand, damaging artifacts are often caused by mis- 
classifications in the segmentation. Generally, however, 
as the rate improves the distortion suffers, and as the 
distortion improves the rate suffers. This is known as the 
rate-distortion compromise. The optimal rate-distortion 
compromise js often a matter of individual preference or 
a function of particular constraints imposed by specific 
applications. 

[0006] In any event, previously developed segmenta- 
tion algorithms or systems, employing so called direct 
segmentation methods, typically compute or determine 
segmentation using only the input image or pixel data. 
They do not consider the properties of the subsequent 
compression technique applied, nor is the rate-distor- 
tion compromise desired by a user considered. That is 
to say, segmentation is not regulated by the ultimate out- 
come of the compression achieved. Rather, the input im- 
age or pixel data is classified for segmentation solely 
based upon a predetermined set of guidelines which de- 
termine classification from the data itself. For example, 
if based on the predetermined guidelines a region of a 
document is determined to contain text, then segmen- 
tation into foreground and background layers and gen- 
eration of a mask layer for a 3-layer MRC model would 
be carried out accordingly regardless of the ultimate ef- 
fect that segmentation may have on the subsequent 
compression. 

[0007] Accordingly, the present invention contem- 
plates a new and improved technique for document or 
image segmentation and compression which over- 
comes the above-referenced problems and others. 
[0008] In accordance with one aspect of the present 
invention, a method of image compression is provided. 
The method includes digitizing an image and segment- 
ing the image in a plurality of different classes to gener- 
ate a digitized image according to a resulting segmen- 
tation obtained from rate and distortion measurements. 
The method further includes determining a bit rate for 
each of the compressed images, and determining how 
much image distortion results from each compression. 
Finally, the manner of segmentation which results in an 
optimal compromise between the rate and distortion is 
selected. 

[0009] In accordance with a more limited aspect of the 
present invention, the segmenting is carried out using a 
3-layer MRC model. However, any of the well known im- 
aging model, known in the art, may be used. 
[0010] In accordance with a more limited aspect of the 
present invention, the step of determining the distortion 
further includes reconstructing the compressed image 
and calculating the distortion from the reconstructed im- 
age and original digitized image. 

[0011] In accordance with a more limited aspect of the 
present invention, the method further includes re-com- 
pressing the digitized image corresponding to the se- 
lected class of segmentation. 

[0012] In accordance with a more limited aspect of the 
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present invention, the method further includes output- 
ting the compressed image corresponding to the select- 
ed manner of segmentation. 

[0013] In accordance with a more limited aspect of the 
present invention, the optimal compromise between the 5 
rate and distortion is achieved when a weighted sum of 
the rate and distortion is at a minimum. 
[0014] In accordance with a more limited aspect of the 
present invention, the weighting of the sum is set to favor 
one of the rate and distortion over the other. 
[0015] In accordance with a more limited aspect of the 
present invention, the weighting of the sum is adjusta- 
ble. 

[0016] In accordance with another aspect of the 
present invention, an image compression system for 
compressing an input image includes a first processing 
bank. The first processing bank includes an array of first 
processors which each have a distinct coder that sepa- 
rately carries out coding to segment and compress the 
image. The first processors each output a bit rate, or an 
estimated bit rate, and image distortion measurement 
resulting from their respective codings. The system also 
includes an optimization engine which receives each 
pair of the rate and distortion measurements from the 
first processors. The optimization engine selects the first 
processor having an optimized compromise between 
the rate and distortion measurements. 
[001 7] I n accordance with a more limited aspect of the 
present invention, the coders of the first processors 
compress the image using a 3-layer MRC model. 
[0018] In accordance with a more limited aspect of the 
present invention, the optimization engine calculates 
the optimized compromise between the rate and distor- 
tion measurements using a cost function which is a 
weighted sum thereof. The optimization engine then se- 
lects the first processor whose associated cost function 
is lowest. 

[001 9] In accordance with a more limited aspect of the 
present invention, the weighting of the sum is adjusta- 
ble. 

[0020] In accordance with a more limited aspect of the 
present invention, each of the first processors further 
includes a decoder which reconstructs the image from 
the coder. The reconstructed image is used as a basis 
for the distortion measurement. 

[0021 ] In accordance with a more limited aspect of the 
present invention, the system further includes a second 
processing bank having an array of coders. The original 
digitized image from the selected first processor is trans- 
mitted to a corresponding coder in the second process- 
ing bank for re-coding. The re-coding is the same as the 
coding carried out by the selected first processor. 
[0022] In accordance with a more limited aspect of the 
present invention, the compressed image from the se- 
lected first processor is output by the system. 
[0023] In accordance with a more limited aspect of the 
present invention, the image compression system is in- 
corporated in a document processing network such that 



the compressed image is routed over the network to an 
output device in compressed form. 
[0024] In accordance with a more limited aspect of the 
present invention, the output device is a printer, a digital 
copier, a xerographic copier, a fax machine, a monitor, 
or a storage device. 

[0025] One advantage of the present invention is that 
it produces more robust segmentations. 
[0026] Another advantage of the present invention is 
that misclassifications which can cause severe artifacts 
are substantially eliminated insomuch as all possible 
segmentation choices are tested for each block of the 
image and the corresponding distortion is measured. 
[0027] Yet another advantage of the present invention 
is that a user can control the trade-off between the bit 
rate and the distortion. 

[0028] Still further advantages and benefits of the 
present invention will become apparent to those of or- 
dinary skill in the art upon reading and understanding 
the following detailed description of the preferred em- 
bodiments. 

[0029] The present invention may take form in various 
components and arrangements of components, and in 
various steps and arrangements of steps. The drawings 
are only for purposes of illustrating preferred embodi- 
ments and are not to be construed as limiting the inven- 
tion. 

FIGURE 1 is diagrammatic illustration of an image 
processing system for use in accordance with as- 
pects of the present invention; 
FIGURE 2 is diagrammatic illustration of a first em- 
bodiment of an image processing unit in accord- 
ance with aspects of the present invention; and, 
FIGURE 3 is diagrammatic illustration of another 
embodiment of an image processing unit in accord- 
ance with aspects of the present invention. 

[0030] With reference to FIGURE 1, a document or 
image processing system A employs an input terminal 
10 to read or otherwise receive an input document or 
image, optionally, a full color document or image. The 
input terminal 1 0 optionally includes one or more suita- 
ble input devices such as an electronically or otherwise 
stored image reader 1 2, a computer image or document 
generator 14, a scanner 16, or the like which reads in 
or generates the input document or image as pixels or 
digital image data. Preferably, the digitized input docu- 
ment or image exists as a file on a computer network or 
a document management/processing network. 
[0031] The image is sampled by an image processing 
unit 100 which carries out segmentation and compres- 
sion. After compression is completed, the compressed 
image or document is output from the image processing 
unit 100. The compressed image is optionally electron- 
ically stored in memory 20 or an otherwise appropriated 
storage device. Alternately, the compressed image or 
document is stored optically or magnetically on a disk 
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or tape. In any event, the compressed image or docu- 
ment is readily accessed and/or transported over the 
network in the usual manner. Of course, prior to render- 
ing by an appropriate rendering engine, such as a print- 
er, display monitor, xerographic or digital copier, fax ma- 
chine, etc., the compressed imaged is first decom- 
pressed or otherwise reconstructed in the usual manner. 
[0032] With reference to FIGURE 2, an exemplary im- 
age processing unit 100 in accordance with aspects of 
the present invention is shown. From the input image 
1 02, a sample 1 04 is taken. As shown, the sample 1 04 
is an 8 by 8 block of pixels. Alternately, other sample 
sizes are employed as desired for particular applica- 
tions. The sample 1 04 is processed by each of a plurality 
of coder-decoder pairs (CODECs) in an array or bank 
110 of CODECs, namely, CODEC_1 through 
CODEC_N. 

[0033] The coders in CODECs 1 through N each per- 
form a distinct coding (i.e., partition blocks into different 
layers and compression) of the sample 104. In a pre- 
ferred embodiment, the segmentation is carried out us- 
ing a 3-layer MRC model. Collectively, the segmenta- 
tions carried out by the coders in bank 1 1 0 represent all 
the possible segmentations available for the different 
classifications or types of blocks the sample 1 04 may 
represent. In this manner each possible segmentation 
is carried out on each sample regardless of the classi- 
fication into which the sample 1 04 would otherwise fall. 
[0034] That is, for example, in a 3-layer MRC model, 
there may be a plurality separate classifications into 
which the sample 1 04 falls. The sample 1 04 may repre- 
sent a first type of block (i.e., it may be predominately 
text and/or line graphics), a second type of block (i.e., it 
may be predominately pictures and/or background) or 
a third type of block (i.e. , a combination of first and sec- 
ond types of blocks). Ordinarily then, the sample 104 
would be segmented in a first manner if it was the first 
type of block, in a second manner if it was the second 
type of block, and yet a third manner if it was the third 
type of block, wherein each of the first, second and third 
segmentations where different from one another. 
[0035] In the 3-layer MRC model, the first type of block 
may be classified on a pixel-by-pixel basis using thresh- 
olding to put the pixels into one of two sets, with extract- 
ed colors from each set being used for all the pixels of 
the respective set. Optionally, the one with the higher 
luminance is the background color, and the other the 
foreground color. The mask layer is then generated to 
indicate which pixels belong in the respective back- 
ground and foreground layers. 

[0036] Conversely, the second type of block may be 
classified by putting the whole block into the background 
layer, setting the foreground layer to be a uniform block 
(e.g., with a mean color the same as a previous fore- 
ground block in raster order), and loading the mask layer 
with 0's to indicate that pixels of this block belong to the 
background layer. 

[0037] As for the third type of block, it may be classi- 



fied by putting the whole block into the foreground layer, 
setting the background layer to be a uniform block (e.g., 
with a mean color the same as a previous background 
block in raster order), and loading the mask layer with 
5 1 's to indicate that pixels of this block belong to the fore- 
ground layer. 

[0038] In this manner, the foreground layer is made to 
contain the colors of text and line graphics, while the 
background layer is made to contain the colors of pic- 

w tures and background. Accordingly, both layers have 
significant redundancy and can be compressed aggres- 
sively. The mask layer is made to contain the contours 
of text and other fine image structures, however, it has 
only two colors and can be compressed effectively using 

15 a binary image compression algorithm, such as, a token 
based compression algorithm. Optionally, other image 
segmentation protocols and/or algorithms as are known 
in the art may be employed. 

[0039] In any event, with respect to the present inven- 
20 tion, the coders of bank 110 are applied to all the seg- 
mentations on each sample 104 regardless of the clas- 
sification or type of block the sample 1 04 is. For exam- 
ple, coder_1 optionally carries out a coding correspond- 
ing to a first type of block, coder_2 carries out a seg- 
25 mentation corresponding to a second type of block, and 
so on. However, regardless of the type of block the sam- 
ple 104 may arguably represent, each sample 104 is 
processed by every coder in the bank 110. 
[0040] In a preferred embodiment, the coders in each 
30 CODEC also carry out compression of the segmented 
image sample. Preferably, the coders employ any 
known compression techniques or algorithms, such as, 
e.g., JPEG, JBIG2 orthe like. Alternately, different com- 
pression algorithms and/or techniques are used for dif- 
35 ferent layers of the segmented image sample. In a pre- 
ferred embodiment, the foreground and background lay- 
ers of the 3-tayer MRC model are compressed using 
JPEG, while the mask layer is compressed using JBIG2. 
[0041] The decoder portions of each CODEC decom- 
40 press and/or reconstruct the image from their corre- 
sponding coders. From each CODEC, the bit rate (R) of 
the compressed imaged and the distortion (D) of the re- 
constructed image are obtained, either via a direct 
measurement or an appropriately calculated estimation. 
45 Optionally, the bit rate is determined from a direct count 
or measure of the bits in the compressed image and a 
direct count or measure of the pixels in the uncom- 
pressed image. In a preferred embodiment, the distor- 
tion measurement is class-dependent. That is, the man- 
50 ner in which the distortion is measured varies depending 
on what the type or classification of block is for which 
the distortion is being determined. This is advantageous 
insomuch as different regions or block types of an output 
image can tolerate different types of distortion. For ex- 
55 ample, errors in high frequency bands can be substan- 
tially ignored in certain types of foreground and back- 
ground blocks, but they can cause severe artifacts in 
other regions, e.g., in text. In any event, each CODEC 
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outputs or otherwise makes accessible their corre- 
sponding set of R and D values, namely, R1 and D1 from 
CODECJ ; R2 and D2 from CODEC_2; through RN and 
DN from CODECJM. 

[0042] The R, D sets from the bank 11 0 of CODECS 5 
are transmitted to, accessed or otherwise received by a 
rate-distortion optimization engine (RDOE) 120. Using 
a cost function (CF), the RDOE 120 calculates or other- 
wise determines which CODEC (and hence which seg- 
mentation) achieved the best rate-distortion compro- 10 
mise. In a preferred embodiment, the cost function is a 
weighted sum of the bit rate and the distortion, and may 
be represented mathematically as: 

15 

CF=R + XD (1); 

where X is a non-negative real number or coefficient 
which represents the weighting factor controlling the 
trade-off between R and D. Alternately, of course, the 20 
weighting factor A. is a coefficient of R rather than D. Nev- 
ertheless, as shown in equation (1 ), relatively higher val- 
ues of X correspond to a relatively higher importance 
placed on the distortion, with respect to rate in the com- 
promise. Alternatively, relatively lower values of X cor- 25 
respond to a relatively higher importance placed on the 
rate, with respect to distortion in the compromise. In the 
case where X is equal to 0, the lowest quality is achieved 
according to the rate measurement. 

[0043] The RDOE 120 evaluates the cost function for 30 
each CODEC or R, D set 1 through N. The CODEC or 
R, D set corresponding to the minimum evaluated cost 
function is identified as the one having the optimum rate- 
distortion compromise. 

[0044] Optionally, as shown in FIGURE 2, X is ob- 35 
tained from a memory 122 or another like storage de- 
vice. The memory 1 22 optionally stores one or more de- 
fault values for X. Alternately, X is stored optically or 
magnetically on a disk or tape. In a preferred embodi- 
ment, A. is user defined or adjusted to achieve a desired *o 
relative weighting between the rate and the distortion. 
Accordingly, an appropriate input device (e.g., key- 
board, mouse, and/or the like) is provided such that the 
desired X value may be entered either directly into the 
RDOE 120 or into the memory 122. 45 
[0045] The RDOE 120 selects the identified CODEC 
from the bank 110 and connects it to a corresponding 
coder in a bank 130 containing coder_1 through 
coder_N. As shown, e.g., CODEC_2 has been identified 
as having to the optimal rate-distortion compromise and so 
as such has been connected to its corresponding 
coder_2 in the coder bank 1 30. In this manner, the orig- 
inal digitized image is output or otherwise transmitted 
from the identified CODEC with the optimum rate-dis- 
tortion compromise to the corresponding coder in bank 55 
130 for re-coding. Preferably, the coders in bank 130 
are identical to their counterparts in the corresponding 
CODECS. Accordingly, the output from the bank 130 of 



coders is a compressed document or image 1 40 having 
the identified optimal rate-distortion compromise. 
[0046] With further reference to FIGURE 3, another 
exemplary image processing unit 100 in accordance 
with aspects of the present invention is shown, similar 
to that shown in FIGURE 2. The most significant differ- 
ence is that the bank 1 1 0a comprises an array of coders 
rather than the CODECS of bank 110. Accordingly, the 
output image from bank 11 0a is not decompressed and/ 
or reconstructed due to the lack of a decoder. The iden- 
tified coder in bank 110a having the optimal rate-distor- 
tion compromise may therefore be selected by the 
RDOE 120 such that its output gives the compressed 
image 1 40 directly without having to use the re-coding 
bank 130. However, in the embodiment of FIGURE 3, 
both the rate and distortion have to be measured or es- 
timated from the compressed image without the benefit 
of reconstruction. 

[0047] In any event, an advantageous compression is 
achieved with an optimized rate-distortion compromise. 
That is to say, via feedback from the resulting com- 
pressed image and/or the reconstructed Image, the seg- 
mentation of the input image is selected and/or regulat- 
ed such that the best possible rate-distortion compro- 
mise is achieved. Unlike previously developed tech- 
niques, the segmentation applied is chosen without re- 
gard for the classification or block type of the input im- 
age. Rather, the segmentation chosen is based on the 
results achieved. Moreover, by employing an optional 
user defined, or selected weighting factor the rate-dis- 
tortion compromise can be customized as desired for 
particular applications. 

[0048] The invention has been described with refer- 
ence to the preferred embodiments. Obviously, modifi- 
cations and alterations will occur to others upon reading 
and understanding the preceeding detailed description. 
For example, preferably, the invention is implemented 
via a software configuration, a hardware configuration, 
or a combination of both. In any event, it is intended that 
the invention be construed as including all such modifi- 
cation and alterations insofar as they come within the 
scope of the appended claims or the equivalents there- 
of. 



Claims 

1 . A method of image compression , said method com- 
prising the steps of: 

(a) digitizing an image; 

(b) segmenting the image in a plurality of differ- 
ent manners to generate a plurality of segment- 
ed images; 

(c) compressing each of the segmented imag- 
es; 

(d) determining a bit rate for each of the com- 
pressed images; 
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(e) determining how much image distortion re- 
sults from each compression; and, 

(f) selecting the manner of segmentation which 
results in an optimal compromise between the 
rate and distortion. 

2. The method according to claim 1 , wherein the seg- 
menting is carried out using a 3-layer MRC model. 

3. The method according to claim 1 , wherein step (e) 
further comprises: 

reconstructing the compressed image; and, 
calculating the distortion from the reconstruct- 
ed image. 

4. The method according to claim 3, said method fur- 
ther comprising: 

(g) re-compressing the reconstructed image 
corresponding to the selected manner of seg- 
mentation. 

5. The method according to claim 1 , said method fur- 
ther comprising: 

(g) outputting the compressed image corre- 
sponding to the selected manner of segmenta- 
tion. 

6. The method according to claim 1 , wherein the opti- 
mal compromise between the rate and distortion is 
achieved when a weighted sum of the rate and dis- 
tortion is at a minimum. 

7. An image compression system for compressing an 
input image, said system comprising: 

a first processing bank, said first processing 
bank including an array of first processors, 
wherein each first processor has a distinct cod- 
er which separately carries out coding to seg- 
ment and compress the image, said first proc- 
essors each outputting a bit rate and image dis- 
tortion measurement resulting from their re- 
spective codings; and, 

an optimization engine which receives each 
pair of the rate and distortion measurements 
from the first processors, said optimization en- 
gine selecting the first processor having an op- 
timized compromise between the rate and dis- 
tortion measurements. 

8. The image compression system according to claim 
7, wherein the coders of the first processors seg- 
ment the image using a 3-layer MRC model. 

9. The image compression system according to claim 



7, wherein the optimization engine calculates the 
optimized compromise between the rate and distor- 
tion measurements using a cost function which is a 
weighted sum thereof, said optimization engine se- 
5 lectin g the first processor whose associated cost 
function is lowest. 

10. The image compression system according to claim 
7, said system further comprising: 



a second processing bank, said second 
processing bank including an array of coders, 
wherein the reconstructed image from the se- 
lected first processor is transmitted to a corre- 
15 spending coder in the second processing bank 

for re-coding, said re-coding being the same as 
the coding carried out by the selected first proc- 
essor. 
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